Papers with network architecture
Meta Learning and Its Applications to Natural Language Processing (2021.acl-tutorials)
Copied to clipboard
| Challenge: | Meta-learning is a new technique that aims to learn better learning algorithms, including better parameter initialization, optimization strategy, network architecture, distance metrics, and beyond. |
| Approach: | This tutorial introduces Meta-learning approaches and the theory behind them, and then reviews the works of applying this technology to NLP problems. |
| Outcome: | This tutorial will introduce Meta-learning approaches and the theory behind them, and then review the works of applying this technology to NLP problems. |
Protein-STORY: Semantic Text-Oriented Representation Yields biologically meaningful Protein embeddings (2026.acl-short)
Copied to clipboard
| Challenge: | Unsupervised representation learning relying on sequence data often overlooks decades of expert-curated biological knowledge stored in textual formats. |
| Approach: | They propose a pipeline that synthesizes protein embeddings from diverse, multi-source text descriptions and a network architecture that integrates high-fidelity functional and structural insights into a unified representation. |
| Outcome: | The proposed pipeline outperforms existing models on diverse downstream tasks (+2 pts F1) and enables zero-shot text-prompted protein search. |
Label Anchored Contrastive Learning for Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | a novel approach to contrastive learning for language understanding is not fully explored . contrastive training has been widely applied to self-supervised representation learning . |
| Approach: | They propose a label anchored contrastive learning approach for language understanding using a class label. |
| Outcome: | The proposed approach improves on GLUE and CLUE benchmarks by 4.1% compared to the state-of-the-art approaches . the proposed approach also improves under the few-shot and data imbalance settings . |
Tensor Product Generation Networks for Deep NLP Modeling (N18-1)
Copied to clipboard
| Challenge: | Using Tensor Product Representations (TPRs) we propose a new architecture for natural language processing based on the principle that hypothesis space for learning includes network hypotheses that are independently known to be suitable for performing the target task. |
| Approach: | They propose a Tensor Product Generation Network (TPGN) which is capable of carrying out TPR computation but uses unconstrained deep learning to design its internal representations. |
| Outcome: | The proposed architecture outperforms baselines on the COCO dataset and can interpret internal representations and operations. |
On the Transferability of Adversarial Attacks against Neural Text Classifier (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that deep neural networks are vulnerable to adversarial examples . a small perturbation to an input alters the model prediction . |
| Approach: | They propose a genetic algorithm to find models that can induce adversarial examples to fool models . they propose word replacement rules that can be used for model diagnostics from these examples . |
| Outcome: | The proposed model can fool almost all existing models, while ignoring the data bias in the training set. |
Glancing Transformer for Non-Autoregressive Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. |
| Approach: | They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup. |
| Outcome: | The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer. |
Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners (2023.findings-acl)
Copied to clipboard
| Challenge: | In Canada, retrieving similar cases and their analysis is a key part of legal work . long processing times are due to a significant backlog and to the amount of work required from counsels . |
| Approach: | They propose to extend existing neural named-entity recognition models to retrieve 19 categories of items from refugee cases. |
| Outcome: | The proposed pipeline achieves a superior F1- score on five of the targeted categories and superior to 80% on an additional 4 categories. |
Latent Group Dropout for Multilingual and Multidomain Machine Translation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Multidomain and multilingual machine translation often rely on parameter sharing strategies, which are hardcoded in the network architecture, independent of the similarities between tasks. |
| Approach: | They propose a method to take advantage of similarities by using a latent-variable model and develop techniques to train this model end-to-end. |
| Outcome: | The proposed model improves translation performance without increasing the model size. |
Layer-Wise Multi-View Learning for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation are limited to the topmost encoder layer’s context representation and cannot perceive the lower encoder layers. |
| Approach: | They propose a layer-wise multi-view learning approach to solve this problem by incorporating an auxiliary view into the model. |
| Outcome: | The proposed model can achieve stable results over multiple strong baselines and is agnostic to network architectures. |
LADIS: Language Disentanglement for 3D Shape Editing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for 3D shape editing produce decoupled, local edits to 3D shapes. |
| Approach: | They propose a tool set that learns latent representations that ground language in 3D geometry. |
| Outcome: | The proposed method outperforms existing SOTA methods by 20% in edit locality and up to 6.6% in language reference resolution accuracy. |
Denoising based Sequence-to-Sequence Pre-training for Text Generation (D19-1)
Copied to clipboard
| Challenge: | PoDA pre-trains encoders and decoders by denoising noise-corrupted text . Unlike encoder-only or decode-only methods, it can be used for text generation tasks without using any task-specific techniques. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) pre-training method PoDA which denoises autoencoders by denoising noise-corrupted text. |
| Outcome: | The proposed method improves model performance over strong baselines without using any task-specific techniques and significantly speed up convergence. |
Matching Distributions between Model and Data: Cross-domain Knowledge Distillation for Unsupervised Domain Adaptation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods require to learn to adapt the target model by exploiting the source data and sharing the network architecture across domains. |
| Approach: | They propose a framework that allows to transfer the knowledge of source domain to the unlabeled target domain without using source data. |
| Outcome: | The proposed framework matches distributions between a trained source model and a set of target data and achieves superior performance on cross-domain text classification. |
Obtaining Faithful Interpretations from Compositional Neural Networks (2020.acl-main)
Copied to clipboard
Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner
| Challenge: | Neural module networks (NMNs) are a popular approach for modeling compositionality but prior work implicitly assumed that the structure of the network modules provides a faithful explanation of the model’s reasoning. |
| Approach: | They propose to use auxiliary supervision to train a model with a structured model that can understand the reasoning process and make better choices for module architecture. |
| Outcome: | The proposed models on two datasets show that the proposed models do not provide a faithful explanation of model behaviour. |
Adversarial Reprogramming of Text Classification Neural Networks (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can cause a machine learning model to misclassify a sample from the classifier's input domain. |
| Approach: | They propose a context-based vocabulary remapping method that performs a computationally inexpensive input transformation to reprogram a victim classification model for a new set of sequences. |
| Outcome: | The proposed method performs a cost-effective input transformation to reprogram a model for a new set of sequences without altering the network architecture or parameters. |
Measure Children’s Mindreading Ability with Machine Reading (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing scoring models do not take the features of the stories and video clips into account when scoring, which will reduce the accuracy of the models. |
| Approach: | They propose to leverage the features extracted from stories and videos related to the questions being asked during the children’s mindreading evaluation. |
| Outcome: | The proposed framework agrees well with human experts on scores produced by the models. |
Deep Neural Model Inspection and Comparison via Functional Neuron Pathways (P19-1)
Copied to clipboard
| Challenge: | a general method for the interpretation and comparison of neural models is proposed . we factor a complex neural model into its functional components . |
| Approach: | They propose a method that factored a complex neural model into its functional components . they use correlated task level and linguistic heuristics to identify correlated pathways . |
| Outcome: | The proposed method can be applied in a purely post-processing manner to understand neural models. |
ARXSA: A General Negative Feedback Control Theory in Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to the self-attention mechanism is proposed for integrating data from multiple batches. |
| Approach: | They propose an autoregressive with exogenous inputs approach for the Transformer model . the proposed method transforms the Encoder block into a negative feedback predictive control system . |
| Outcome: | The proposed method is validated through comparative evaluations. |
Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is a method for extracting facts from text in structured format . alternative formulations allow for longer tuples, but most work focuses on binary predicates only. |
| Approach: | They propose to extract facts from natural language text and represent them as structured triples . they compare different neural network architectures and training approaches . |
| Outcome: | The proposed approach improves the currently best models on the OIE16 benchmark by 0.421 F1 score and 0.420 AUC-PR . |